Skip to content

perf(release-prepare): plan, prep, post and the signing setup as code - #1624

Merged
potiuk merged 3 commits into
perf/release-mgmt-4-promote-announcefrom
perf/release-mgmt-5-prepare
Oct 16, 2026
Merged

perf(release-prepare): plan, prep, post and the signing setup as code#1624
potiuk merged 3 commits into
perf/release-mgmt-4-promote-announcefrom
perf/release-mgmt-5-prepare

Conversation

@potiuk

@potiuk potiuk commented Oct 13, 2026

Copy link
Copy Markdown
Member

Summary

Fifth PR of the release-management stack; it builds on #1618.

  • Scripts move. release-prepare's three skill-local scripts (prev_tag, next_dev_version, category_x) move unchanged into tools/release-management.
  • New commands cover what the agent did by hand:
    • verdict: Step 0 train coverage.
    • plan-query / plan: the planning issue.
    • versions, notice and changelog (with the 90% coverage check).
    • archive-mode / archive-review: the 2e source-archive review.
    • prep-pr and post-pr.
    • signing-eligibility / signing-render: the automated-signing A1 gate and the A4/A5 drafts.
  • Size. SKILL.md goes from 5,532 to 5,027 tokens, and prep.md from 290 to 209 lines; the 2e review moves to archive-review.md at the skill root.

Security notes

  • PR titles are external content. They are rendered as inert text in the planning issue and the changelog: HTML is escaped and line breaks are folded. A title holding <!-- … --> can no longer be hidden from GitHub's renderer while a later agent still reads it.
  • The 2e listings now write to $TMPDIR, not /tmp, which broke under the sandbox.
  • signing-render refuses key material or a signing step. Step A still never files the Infra ticket, sends mail or adds a key.

Behaviour changes worth a look

  • Step 0 is fail-closed. It blocks without GitHub access, on a planning issue missing its label, and on a release line it cannot read until --train-confirmed is passed.
  • Changelog categories.
    • A new optional key, changelog_label_categories (LABEL=SECTION, documented in the config template), maps labels to sections. Unset, a label is sorted by its words.
    • prep-pr refuses coverage below 90% until --assign classifies the rest, matching the existing "ask the RM to classify before the PR is opened".
  • Planning-issue body: groups are sorted alphabetically with unlabelled PRs last, and numbering runs across groups.

Type of change

  • Skill change — eval fixtures updated below
  • Python package

Test plan

  • prek run --all-files passes on the stack tip.
  • tools/release-management: pytest / ruff / mypy, including the moved script tests and a real-load integration test.
  • Evals: the scripted cases are now pytest cases named after each former case. They are step-0 1/2/4, step-1-plan 1/2, step-2-prep 1/2, and the whole step-14-post suite. The kept judgement cases pass 6/6, against a baseline of 15/15.
  • non-asf-profile-smoke: both of its prepare cases pass. Two of its other cases fail on main independently of this stack; see the notes below.

Notes for reviewers

non-asf-profile-smoke has two failures that reproduce on runs without this change:

Both are follow-ups outside this stack.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5). I reviewed the diff and the test results; commits carry Generated-by:.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv

@potiuk potiuk added family:setup setup-* skills family:tools tools/* capability:resolve Close-out: invalidate, dedupe, CVE allocate, announcement family:release-management release-* skills substrate:release Tool substrate: release-artefact helpers (reproducible archive build, lint, comparison) labels Oct 13, 2026
@potiuk
potiuk added this pull request to stack #1620 October 13, 2026 15:24
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch from 383a723 to 8aefa45 Compare October 14, 2026 08:09
@github-actions github-actions Bot added family:docs Docs, MISSION.md, READMEs substrate:framework-dev Tool substrate: build / validate / eval the framework itself labels Oct 14, 2026
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch 2 times, most recently from 8473ef7 to 842f8dd Compare October 15, 2026 23:37
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch from 842f8dd to 874c60b Compare October 15, 2026 23:47
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch 2 times, most recently from 8462f4a to c4ee37c Compare October 16, 2026 00:25
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch from c4ee37c to c6fc2a5 Compare October 16, 2026 03:18
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch 2 times, most recently from 003feef to f2494ce Compare October 16, 2026 05:42
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch from f2494ce to f0825e9 Compare October 16, 2026 06:28
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch 2 times, most recently from 1c63581 to e2eca83 Compare October 16, 2026 07:15
potiuk added 3 commits October 16, 2026 08:37
@potiuk
perf(release-prepare): plan, prep, post and the signing setup as code
a1ab2c5 
release-prepare had the agent decide train coverage, render the
planning issue, group the changelog, classify what the source archive
ships and compose every PR by hand. Its three skill-local scripts
(prev_tag, next_dev_version, category_x) move unchanged into
tools/release-management, and new commands do the rest, printing JSON
and running nothing:

- `verdict` (Step 0): a release line must cover the version, plus the
  planning issue, its label and GitHub access; fail-closed.
- `plan-query` / `plan`: the tag-date read and the quoted PR search,
  then the empty-set hand-off or the issue title and body with PRs
  grouped by label.
- `versions`, `notice`, `changelog`: the version per manifest, removed
  NOTICE paragraphs from the diff (whether a removal is justified stays
  the RM's call), and the changelog by section with the 90% coverage
  check (`--assign` for the rest).
- `archive-mode` / `archive-review` (2e): full review, drift check or
  skipped; the bucket table, never-excludable and referenced paths
  refused, symlink chains, the `git grep` loop, the before/after
  listing commands, which now write to $TMPDIR instead of /tmp.
- `prep-pr`, `post-pr`: titles, bodies and scope as a set difference.
- `signing-eligibility` / `signing-render`: the automated-signing A1
  gate and the A4 workflow + A5 config diff, refusing key material.

PR titles are external content: they are rendered as inert text
(HTML escaped, line breaks folded) in the planning issue and the
changelog, so a title cannot hide a comment from GitHub's renderer
while a later agent still reads it. A new optional
`changelog_label_categories` key is documented in the config template.

SKILL.md 5,532 → 5,027 tokens; prep.md 290 → 209 lines, with the 2e
review in archive-review.md at the skill root. Golden rules, Hard
rules, the eval headings and the generated blocks are unchanged.

Evals: the scripted cases (step-0 1/2/4, step-1-plan 1/2, step-2-prep
1/2, the whole step-14-post suite) become pytest cases named after each
former case; the judgement cases stay, their fixtures now carrying the
real tool output.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv
@potiuk
fix(release-prepare): keep LICENSE and NOTICE in the archive and stop…
5fdbc7d 
… silent passes

Review findings on the prepare tool:

- The source-archive review refuses a glob --exclude, and any entry that
  would strip LICENSE / NOTICE / DISCLAIMER, a voter input, a build
  descriptor, a lock or a version manifest; a committed export-ignore that
  already strips one is reported as a violation.
- A path passed on the command line that does not exist is an error, not
  an empty listing; drift mode lists symlinks too.
- Every proposed or added exclusion is reference-checked before the
  .gitattributes block is rendered (refs_unchecked).
- NOTICE is diffed from both refs, not from the compare API, which drops
  files on large releases.
- prep-pr and post-pr require --files; prep-pr requires --archive-mode
  and refuses a skipped review when the mode is full; post-pr refuses to
  bring the changelog into scope.
- --train-confirmed also overrides a release line that names only other
  trains.
- The automated-signing workflow refuses any signing or key-import step
  in build_command, and eligibility follows
  release_process.automated_signing instead of the organization name.
- The planning-issue placeholder check catches placeholders with dots.

Generated-by: Claude Opus 5
@potiuk
fix(release-prepare): protect configured voter inputs and settle refe…
1401856 
…rence checks

Adversarial-review findings on the prepare tool:

- release-config load now carries rat_config_path, rat_excludes_path and
  source_tree_validators in `build`, so the archive review protects the
  configured voter inputs through the documented --load flow.
- Reference scans accumulate across rounds instead of overwriting the
  previous results, so adding an exclusion no longer unchecks the others.
- Refusals are decided before references are weighed, and the excluded
  set is recomputed until it stops changing, so a path referenced only by
  a refused (kept) file is no longer proposed.

Generated-by: Claude Opus 5
@potiuk
potiuk force-pushed the perf/release-mgmt-5-prepare branch from e2eca83 to 1401856 Compare October 16, 2026 07:37
@potiuk
potiuk merged commit a75734c into main Oct 16, 2026
12 checks passed
@potiuk
potiuk deleted the perf/release-mgmt-5-prepare branch October 16, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

capability:resolve Close-out: invalidate, dedupe, CVE allocate, announcement family:docs Docs, MISSION.md, READMEs family:release-management release-* skills family:setup setup-* skills family:tools tools/* substrate:framework-dev Tool substrate: build / validate / eval the framework itself substrate:release Tool substrate: release-artefact helpers (reproducible archive build, lint, comparison)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@potiuk